1. Field of the Invention
This invention relates to electronic messaging systems.
2. Background Art
With the proliferation of personal computers and communications networks such as the internet, electronic mail, commonly referred to as "e-mail," has become a popular mechanism for the exchange or distribution of information among individuals, and within or between enterprises, for both private and commercial purposes. There are a number of disadvantages of current e-mail systems. One disadvantage is that a computer system is needed. Another disadvantage is that it is difficult to use if a sender wants to attach pictures or audio messages to an e-mail message. These disadvantages can be understood by reviewing the way e-mail works.
An e-mail message may be analogized to a posted letter or piece of mail. However, instead of a physical object that is itself physically transported from a sender to a recipient, an e-mail message is an electronic representation that is communicated electronically through a communications network. Examples of communications networks used for communicating e-mail messages include, but are not limited to, telecommunications networks, wide area networks (WANs), local area networks (LANs), the Internet, intranets, extranets, wireless networks, and other networks over which electronic, digital, and/or analog data may be communicated.
Prior Art E-mail Systems
E-mail messages are created, sent, received, and read using a communications program, often referred to as a "mail" or "e-mail" application program. An interface of one prior art e-mail program is illustrated in FIG. 2. The example of FIG. 2 is the interface of the mail module of Netscape Communicator, an internet browser application program. The interface is displayed in a window 200. The interface includes a row of control buttons 201-210 arranged along the top of window 200, column heading boxes 212-215 arranged below control buttons 201-210, and a display field 217 that displays a list of messages. In the example of FIG. 2, the list indicates a single message 216. The information displayed for message 216 in display area 217 includes an icon 218 indicating that message 216 is an e-mail message, text 219 indicating the subject of message 216, namely "Meeting on the 20th", text 220 indicating the source or destination of message 216, in this case "Sender" and text 221 indicating the time that message 216 was sent or received, in this case 9:20 PM.
Control buttons 201-210 are used to create, modify and manipulate messages. Button 201, "Get Msg" is used to display a message that has been selected from the list of messages displayed in display area 217 (messages can also be opened by double clicking on them). Button 202, "New Msg" is used to generate a new message form that can be completed and mailed to a recipient. "Reply" button 203 is used to generate a reply form for replying to a message that is currently in view. When activated, a message form is generated that has as its address the address of the sender of the message being viewed. Optionally, the reply message may include the entire text of the sender's message. "Forward" button 204 generates a message form that includes the sender's message, but with a blank address, so that the message may be optionally annotated and forwarded to another recipient.
Button 205, "File", is used to save a message into a file in a text or other format. Button 206, "Print", is activated to print the message on an attached printer. "Security" button 208 activates security options for a message such as, for example, encryption, or the use of a digital certificate or digital signature. Messages can be deleted using "Delete" button 209. "Stop" button 210 is used to interrupt or stop operations.
Column heading boxes 212-215 contain column headings for the information displayed in display area 217. The headings specified in column heading boxes 212-215 are "Subject", "To/From", "Date", and "Priority", respectively. Messages in the list can be sorted by subject, by sender or recipient, by ascending or descending date, by priority, or by any combination thereof.
An e-mail message form 300 generated using the example e-mail program of FIG. 2 is illustrated in FIG. 3. E-mail message form 300 includes a palette of control buttons 301-308, an address entry area 309, a subject entry area 310, and a message editing area 311. Address entry area 309 indicates the destination(s) for the message, in this case "Receiver@receiver.com". Subject entry area 308 indicates the subject of the message, here "Meeting on the 20th". The body 311 of the message is displayed in editing area 312.
"Send" button 301 is activated to initiate the transmission of the message from the sender to the recipient. "Quote" button 302 is used to insert text from another message into body 311 of message displayed in editing area 312. "Address" button 303 prompts the sender to enter an address of the recipient or to select an address from a stored address book. "Spelling" button 305 performs a spell check on the message text, and "Save" button 306 is used to save a message as a text file.
"Attach" button 304 is used to attach one or more electronic files to the e-mail message. The operation of this e-mail file attachment feature illustrates disadvantages and limitations of the prior art. Often a sender wishes to send one or more files to a recipient. One method of sending a file to a recipient is to copy the information from the file (e.g. the text from a text file) and paste that information into the body 309 of an e-mail message. If the file is large, this may not be possible, since some e-mail programs have limitations on the size of the body of an e-mail message. In other cases, the files represent non-text data, such as sound, images, or movies, for example, that cannot be easily pasted into the body of an e-mail message. In such circumstances, the file attachment feature is used.
When "Attach" button 304 is activated, a dialogue box appears that allows the send er to navigate through a file system and select files to be attached to the e-mail message. After one or more files are selected, the sender causes the e-mail program to transmit the e-mail message and attached file(s) to a recipient. When the recipient reads the message, the reader's e-mail program displays an indicator indicating that one or more files are attached. If the recipient uses the e-mail program of FIGS. 2 and 3 the recipient may retrieve the attached file(s) by activating the "Attach" button. Upon activating the "Attach" button, the recipient is presented with a dialogue box that enables the recipient to retrieve the attached file or files and store them in the recipient's file system.
Attaching Image Files to E-mail Messages
Users often desire to send one or more pictures with an e-mail message. These pictures can be photographs, digital photographs, computer drawings, graphic images, or any other type of image. These kinds of image files often require preparatory work to make them suitable for attachment to e-mail messages. Accordingly, attaching image files to an e-mail message can be a complex process.
Before an image file can be attached to an e-mail message, the file must be present on the computer, or must be placed on the computer. If the file is already located on the computer the user must remember what the file is named, and where it is located. Often it is necessary not only locate the file but also to view it first to make sure that it is indeed the correct file that is intended to be sent. The file must be in a format that the person receiving the file has a capability to use. If it is not the file may have to be converted, using a conversion program, to a common format (such as GIF or JPEG) that the person receiving the file has the capability of viewing.
If the file is not already on the computer it must be taken from another source and placed on the computer. There are a variety of sources for such files. For example, the image file may come from a picture taken by a digital camera, from files on the internet, or from a CD ROM.
If the image is not already in a computer readable form, (for example, if the image is a photographic print or a newspaper clipping) it must first be converted into such a form. This can be done, for example, by using a process called scanning. When an image from a photograph or newspaper article is scanned it is placed on a device called a scanner which is attached to a computer. A scanner illuminates consecutive small sections of the item to be scanned and converts the small section into a series of numbers that represent the colors and intensity at discrete points of the section. The computer stores these numbers in a file, and then the scanner moves on to the next small section of the item to be scanned and the process is repeated over and over until the entire item has been scanned and converted into a sets of numbers and placed in the computer file. The file can then be used to recreate an image of the item scanned on another computer. However, because raw scanned files are often large and therefore difficult to transmit, they are often converted into more compact files (such as GIF or JPEG) which are smaller and therefore more easily transmitted. Scanned files are typically converted using conversion programs, and then stored in their smaller converted form on the hard disk of the computer.
Once a file exists in an appropriate format on the computer it can be sent in the form of an e-mail attachment. To attach an image file to an e-mail message a user utilizes the attachment feature of the user's e-mail program, for example, by clicking the "Attach" button 304 of the e-mail program of FIG. 3. Typically, a dialog box appears with a representation of the file system hierarchy of the computer. The user then navigates through the file system hierarchy to find the desired image file. (The user may need to first locate and view the file to verify that it is the correct file.) The user then attaches the file, usually by double clicking the mouse button on the name of the file. At that point the file has been attached to the e-mail and is ready to be sent. The user then activates the "send" command of the user's e-mail program (e.g. by clicking on the "send" button 301 of FIG. 3), and the e-mail along with the attachment is transmitted.
FIG. 4 illustrates a flow diagram of the process of inputting and attaching an image file to an e-mail as an e-mail attachment using an e-mail program of the prior art. Referring to FIG. 4, at step 401 the image resource file to be sent as an e-mail attachment is selected. At step 403 a determination is made as to whether the selected image file is present in the computer. If the image file to be sent as an attachment is already present on the computer, the process proceeds to step 405 where the image to be sent is located. After the image is located the process proceeds to decision block 407 at which a determination is made as to whether the image is in the correct format. If it is determined at decision block 407 that the image file is in the correct format, the process proceeds to step 411. If the image is not in the correct format, the process proceeds to step 409 where the image is converted to the correct format. The process then proceeds to step 411.
At step 411 the image file in the correct format is attached to the e-mail message. The sequence then proceeds to step 413, where the message containing the attached image file is transmitted to the recipient.
If it is determined at decision block 403 that the image file to be sent as an attachment is not already present on the computer, the process proceeds to decision block 415. At decision block 415 a determination is made as to whether the image is in a computer readable file format. If the image is in a computer readable file format (e.g., on a CD-ROM or retrievable from the internet), the process proceeds to step 417, where the image is retrieved from the appropriate source. The process then proceeds to decision block 407.
If it is determined at decision block 415 that the image is not in a computer readable form, the process proceeds to step 419. At step 419, the image is converted to computer readable form, for example by scanning in the image with a scanner. The process then proceeds to step 421 where the scanned image is converted into a usable format suitable for the recipient who will receive the file as an attachment. The process then proceeds to step 423. At step 423 the scanned and converted image is stored as a file on the hard disk. The process then proceeds to step 411.
Attaching a Sound File to an E-mail Message
Attaching a sound file to an e-mail message is as complex as attaching an image file. Before a sound file can be attached to an e-mail message the sound file must be present on the computer, or must be placed on the computer. If the file is already located on the computer the user must remember what the file is named, and where it is located. Often it is necessary not only locate the file but also to listen to it first to make sure that it is indeed the correct file that is intended to be sent. The file must be in a format that the person receiving the file has a capability to use. If it is not the file may have to be converted, using a conversion program, to a common format (such as WAV) that the person receiving the file has the capability of playing.
If the sound is not already in a computer readable file format, (for example if it is a spoken message, or is analog data contained on an audio tape) it must first be converted to a computer readable form. This can be done, for example, by using a microphone and a sound card installed in the computer for this purpose. To input a sound message into the computer the user will typically activate a sound recording program. The user will then activate a "record" function within the sound recording program. When the recording function has been activated the user speaks the message, plays the audio tape containing the message or otherwise creates the sound that is to be recorded. When the message is completed the user activates the "stop recording" function of the sound recording program. The microphone and sound card combination convert the sound that is entering the microphone while the recording function is active into a digital format that can be stored on the computer. One process for doing so is known as "sampling".
The process of sampling can be reversed and the sampled digital sound data converted back into sound by recreating the intensity and frequency of the sound represented by the stored digital data. To do so a playback program is typically used. The playback program takes the digital sound file that the computer has stored and converts it back to the original sound through the use of a sound card, amplifier and speaker within the computer.
If the sampling process is not rapid enough (e.g. if it does not meet the well known "Nyquist" criteria), the quality of the sound reproduced from the digital sound file will not be very high, and the sound will be distorted. To keep this sampling rate high (typically tens of thousands of samples per second) the computer must store tens of thousands of numbers per second to accurately reproduce a sampled sound. A file representing a few seconds of sound can therefore easily grow to be quite large.
It is because of this large file size that sound files are not usually stored as raw files, but, like large image files, are encoded to make their size smaller. A sound encoding program is used for this purpose. A sound encoding program can take advantage of the fact that sound in general and speech in particular has patterns of frequency and intensity, and use those patterns to reduce the size of the file. There are many different types of encoding used in sound, the WAV file encoding being one of the most popular.
After the sound has been recorded by the computer it usually is listened to by the user to ensure that the correct sound clip is contained within the file. Typically though the file will contain some unintended sounds at the beginning or end, or may not contain the entire sound message intended to be recorded. If the entire message is not contained within the file the normal procedure is to re-record the message. If the file contains additional sounds the file can be edited using a sound editing program. Usually a sound editing program is included as part of a recording program.
A sound editing program usually generates a visual representation of the sound in the form of a linear waveform graph. Several seconds of sound can typically be displayed on the screen at a time. When the sound file is played using the sound editing program, the waveform is highlighted to show the part of the waveform that is currently being heard. The user of the program can then select and delete the undesired portions of the sound file. In that way a user can, for example, eliminate a preliminary clearing of a throat prior to the speech intended to be recorded, or perhaps the beginning of a next sentence that was accidentally recorded.
After the file has been properly edited it can be stored on the computer's hard disk under a suitable name. It can also be converted, if necessary to a common format such as WAV that the intended recipient of the file has the capability of playing.
Once the file exists in an appropriate format on the computer it is ready to be sent as an e-mail attachment.
The flow process of inputting and attaching a sound file to an e-mail message is similar to the process of attaching an image file to an e-mail message, described above with reference to FIG. 4. The user must locate the file or import the file to be attached, playback the file to ensure it contains the intended sound clip, put it in the correct format, and then attach it to an e-mail message.
Receiving and Decoding E-mail Attachments
When a user receives an e-mail message that includes audio and/or image attachments, a complex series of operations must be performed before the recipient can access the attached files. First, the recipient must determine whether the recipient has received an e-mail message containing image or sound file attachments. The recipient looks for new messages using a mail checking function of the user's e-mail program, for example by clicking on the "Get Msg" button 201 of the e-mail program of FIG. 2. Identifying information for any new message that has been received is added to the message window on the recipient's display. For example in FIG. 2, the identifying information "Meeting on the 20th" for message 216 appears in the message window. (If the message is unread it sometimes will appear in boldface type).
To find out if the e-mail message contains an attachment the user must open the e-mail message. For example, the e-mail may be opened by clicking twice on the subject of the message. The message will then open up in a window as shown, for example, in FIG. 3. If there is an attachment an attachment indicator is typically displayed, such as, for example, an icon at the end of the message.
Once the user has determined that there is an attachment to the e-mail message, the user must retrieve and save the attachment. To save an attachment the attachment must first be selected. This may be done, for example, by clicking on the attachment icon. The user must then specify where the attachment is to be saved. Typically, after the attachment has been selected, a dialog box appears that allows the user to navigate through the file system and select where the attached message will be saved. The same dialog box will usually also allow the user to change the name of the attached file if desired.
After the attached file has been saved, the user may close or delete the e-mail message. To view or listen to the attached file, the user must first navigate through the file hierarchy to find the correct file and determine whether it is in a format compatible with the user's computer system. If the file is not in a format that is compatible with the programs that are present in the user's computer, the user may need to convert the file using a conversion program and then save the converted file. If the file is not in a usable format, and if the proper conversion software to convert it to the correct format is not available, the user may either obtain the necessary software, or simply discard the file as useless.
Once the file is in the proper format the user then may access (view or listen to) the file. To access the file the user must choose an appropriate accessing program and activate it. Once the accessing program has been activated the user must again navigate through the file system hierarchy to retrieve the desired file.
Because separate programs are typically required to read an e-mail message and to access attached files it is difficult to read a message and access an attached file at the same time. To do so both the e-mail program and the access program must be displayed on the user's computer monitor at the same time. Even if both programs are displayed at the same time usually only one can be active. However, to read different parts of the e-mail message the e-mail program must be the active program, while to move, adjust, or re-size an image, or to listen to a sound attachment file, the image or sound accessing program must be the active program. Often the only practical solution is to print out the text of an e-mail message on a computer, and then with the print out of the message in hand access the attached file on the computer using the appropriate accessing program.
One can see that for both sound and image file attachments the process of obtaining the file, converting the file to the proper format and using an accessing program to access the file can take many steps and be quite time consuming. In addition to being quite time consuming there is no guarantee that a user will be able to decode an attached file, either because of lack of proper decoding hardware or software, or because the process can often be so complex that the user may lose interest and simply decide it is not worth the trouble.
Other Disadvantages of E-Mail
The use of e-mail requires a certain level of computer knowledge and familiarity. An e-mail program can be quite overwhelming if the user is a person who is easily intimidated by computers. It can also be confusing to computer literate individuals who have not had a great deal of e-mail experience or are presented with an e-mail program that they are not familiar with.
The addresses of e-mail messages also can be quite confusing. Instead of having an easily understood form such as "Aunt Matilda from Seattle" e-mail addresses often take arbitrary and confusing forms such as "1654256@compuserve.com". Because received e-mail messages are usually identified by the e-mail address of the sender, an e-mail recipient will often have to actually open a message to determine who it is from. Even if the e-mail user successfully opens the mail message, the user may be confronted with a message that is contained wholly or partly in an attached file. The attached file may have a voice message within it, or a picture, or video that requires activation of a special accessing program before the entire message can be delivered.
E-mail also sometimes presents a problem in the manner that e-mail messages are displayed. For example, one user may send another user a message in a font that, while readable on the sender's machine, is difficult to read on the recipient's machine. This can be the case even if each user's e-mail program is from the same company, and the version is different. For example, an e-mail message sent by a user using a Japanese version of Microsoft Exchange may be unreadable when sent to a user using an American version of Microsoft Exchange.